DAC Guidelines and Reference Series 


Quality Standards for 
Development Evaluation 



«• 

OECD 


Copyi filled material 





Copyrighted material 


DAC Guidelines and Reference Series 


QUALITY STANDARDS 
FOR DEVELOPMENT EVALUATION 


OECD 

DEVELOPMENT ASSISTANCE COMMITTEE 



Copyrighted material 


ORGANISATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT 


The OECD is a unique forum where the governments of 30 democracies work together to 
address the economic, social and environmental challenges of globalisation. The OECD 
is also at the forefront of efforts to understand and to help governments respond to new 
developments and concerns, such as corporate governance, the information economy and the 
challenges of an ageing population. The Organisation provides a setting where governments 
can compare policy experiences, seek answers to common problems, identify good practice 
and work to co-ordinate domestic and international policies. 

The OECD member countries are: Australia, Austria, Belgium, Canada, the Czech Republic, 
Denmark, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Japan, Korea, 
Luxembourg, Mexico, the Netherlands, New Zealand, Norway, Poland, Portugal, the Slovak 
Republic, Spain, Sweden, Switzerland, Turkey, the United Kingdom and the United States. The 
Commission of the European Communities takes part in the work of the OECD. 


This work is published on the responsibility of the Secretary-General of the OECD. The 
opinions expressed and arguments employed herein do not necessarily reflect the official 
views of the Organisation or of the governments of its member countries. 


You can copy, download or print OECD content for your own use, and you can include excerpts from OECD publications, 
databases and multimedia products in your own documents, presentations, blogs, websites and teaching materials, 
provided that suitable acknowledgment of OECD as source and copyright owner is given. All requests for public or 
commercial use and translation rights should be submitted to rights@oecd.org. Requests for permission to photocopy 
portions of this material for public or commercial use shall be addressed directly to the Copyright Clearance Center 
(CCC) at info@copyright.com or the Centre frangais d’exploitation du droit de copie (CFC) at contact@cfcopies.com. 

ISBN 978-92-64-08390-5 (PDF) 


©OECD 2010 


THE NETWORK ON DEVELOPMENT EVALUATION 


The Network on Development Evaluation is a subsidiary body of the Development Assistance 
Committee (DAC) at the OECD. Its purpose is to increase the effectiveness of international 
development programmes by supporting robust, informed and independent evaluation. The 
Network is a unique body, bringing together 32 bilateral donors and multilateral development 
agencies: Australia, Austria, Belgium, Canada, Denmark, the European Commission, Finland, 
France, Germany, Greece, Ireland, Italy, Japan, Korea, Luxembourg, the Netherlands, New 
Zealand, Norway, Portugal, Spain, Sweden, Switzerland, the United Kingdom, the United 
States, the World Bank, the Asian Development Bank, the African Development Bank, the 
Inter-American Development Bank, the European Bank for Reconstruction and Development, 
the United Nations Development Programme, and the International Monetary Fund. 

For further information on the work of the DAC Evaluation Network, please visit the website 
www.oecd.org/dac/evaluationnetwork or email dacevaluation.contact@oecd.org 


Also available in French under the title: 

Normes de qualite pour revaluation du developpement 





EVALUATION IN AN EVOLVING DEVELOPMENT CONTEXT 

quality evaluation should provide credible and useful evidence to strengthen 
accountability for development results or contribute to learning processes, or 
both. These Standards aim to improve quality and ultimately to reinforce the 
contribution of evaluation to improving development outcomes. 

New aid modalities, consensus on shared development goals and the commitments 
made in the Paris Declaration on Aid Effectiveness (2005) and the Accra Agenda for 
Action (2008) are changing the way development partners address global challenges. 
In this evolving development context, evaluation has an important role in informing 
policy decisions and helping to hold all development partners mutually accountable 
for development results. The way development evaluation is carried out must also 
reflect this new context, becoming more harmonised, better aligned and increasingly 
country-led, to meet the evaluation needs of all partners. 

The DAC Quality Standards for Development Evaluation reflect this evolving 
framework and provide a guide to good practice in development evaluation. Built 
through international consensus, the Standards are intended to serve as an incentive 
and inspiration to improve evaluation practice. 

The draft Standards were approved for a three-year test phase in 2006 and have 
been revised based on experience. A range of development partners have contributed 
to this process, including donors and partner countries. Initial input was provided 
during a workshop in New Delhi in 2005. A 2008 survey of the use of the Standards, 
a 2009 workshop held in Auckland and comments submitted by the members of the 
DAC Network on Development Evaluation, helped to improve and finalise the text, 
with support from the Secretariat of the OECD. The Standards were approved by the 
DAC Network on Development Evaluation on 8 January 2010 and endorsed by the 
DAC on 1 February 2010. 



Nick York 

Chair of the OECD DAC Network on Development Evaluation 
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Terms used in this document 

The term ‘development intervention’ is used in the Standards as a general term for 
any activity, project, programme, strategy, policy, theme, sector, instrument, modality, 
institutional performance, etc, aimed to promote development. 

The term ‘evaluation report’ is used to cover all evaluation products, which may take 
different forms, including written or oral reports, visual presentations, community 
workshops, etc. 


INTRODUCTION 


The DAC Quality Standards for Development Evaluation identify the key pillars needed for a 
quality development evaluation process and product. They are intended for use by evaluation 
managers and practitioners. The Standards are not mandatory, but provide a guide to good 
practice. They were developed primarily for use by DAC members, but broader use by all 
other development partners is welcome. 

The Standards aim to improve quality and ultimately to strengthen the contribution of 
evaluation to improving development outcomes. Specifically, the Standards are intended to: 

improve the quality of development evaluation processes and products, 

facilitate the comparison of evaluations across countries, 

support partnerships and collaboration on joint evaluations, and 

increase development partners’ use of each others’ evaluation findings. 

The Standards support evaluations that adhere to the DAC Principles for the Evaluation 
of Development Assistance (1991), including impartiality, independence, credibility and 
usefulness, and should be read in conjunction with those principles. The Principles focus on 
the management and institutional set up of evaluation systems and remain the benchmark 
against which OECD DAC members are assessed in DAC Peer Reviews. By contrast, the 
Standards inform evaluation processes and products. The Standards can be used during the 
different stages of the evaluation process and in a variety of ways, including to assess the 
quality of evaluations, inform practice, strengthen and harmonise evaluation training, or as 
an input to create evaluation guidelines or policy documents. 

The Standards should be applied sensibly and adapted to local and national contexts and 
the objectives of each evaluation. They are not intended to be used as an evaluation manual 
and do not supplant specific guidance on particular types of evaluation, methodologies or 
approaches. Further, these Standards do not exclude the use of other evaluation quality 
standards and related texts, such as those developed by individual agencies, professional 
evaluation societies and networks. 

This document is structured in line with a typical evaluation process: defining purpose, 
planning, designing, implementing, reporting, and learning from and using evaluation 
results. The Standards begin with some overall considerations to keep in mind throughout 
the evaluation process. An annex provides references to related OECD DAC development 
evaluation publications. 


1 OVERARCHING CONSIDERATIONS 


1.1 DEVELOPMENT EVALUATION 

Development evaluation is the systematic and objective assessment of an on-going 
or completed development intervention, its design, implementation and results. In the 
development context, evaluation refers to the process of determining the worth or 
significance of a development intervention. 

When carrying out a development evaluation the following overarching considerations 
are taken into account throughout the process. 

1.2 FREE AND OPEN EVALUATION PROCESS 

The evaluation process is transparent and independent from programme management 
and policy-making, to enhance credibility. 

1.3 EVALUATION ETHICS 

Evaluation abides by relevant professional and ethical guidelines and codes of 
conduct for individual evaluators. Evaluation is undertaken with integrity and honesty. 
Commissioners, evaluation managers and evaluators respect human rights and 
differences in culture, customs, religious beliefs and practices of all stakeholders. 

Evaluators are mindful of gender roles, ethnicity, ability, age, sexual orientation, 
language and other differences when designing and carrying out the evaluation. 
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1.4 PARTNERSHIP APPROACH 

In order to increase ownership of development and build mutual accountability 
for results, a partnership approach to development evaluation is systematically 
considered early in the process. The concept of partnership connotes an inclusive 
process, involving different stakeholders such as government, parliament, civil society, 
intended beneficiaries and international partners. 

1.5 CO-ORDINATION AND ALIGNMENT 

To help improve co-ordination of development evaluation and strengthen country 
systems, the evaluation process takes into account national and local evaluation 
plans, activities and policies. 

1.6 CAPACITY DEVELOPMENT 

Positive effects of the evaluation process on the evaluation capacity of development 
partners are maximised. An evaluation may, for instance, support capacity development 
by improving evaluation knowledge and skills, strengthening evaluation management, 
stimulating demand for and use of evaluation findings, and supporting an environment 
of accountability and learning. 

1.7 QUALITY CONTROL 

Quality control is exercised throughout the evaluation process. Depending on the 
evaluation’s scope and complexity, quality control is carried out through an internal 
and/or external mechanism, for example peer review, advisory panel, or reference 
group. 




2.1 RATIONALE AND PURPOSE OF THE EVALUATION 

The rationale, purpose and intended use of the evaluation are stated clearly, addressing: 
why the evaluation is being undertaken at this particular point in time, why and for 
whom it is undertaken, and how the evaluation is to be used for learning and/or 
accountability functions. 

For example the evaluation’s overall purpose may be to: 

contribute to improving a development policy, procedure or technique, 

consider the continuation or discontinuation of a project or programme, 

account for public expenditures and development results to stakeholders and 
tax-payers. 

2.2 SPECIFIC OBJECTIVES OF THE EVALUATION 

The specific objectives of the evaluation clarify what the evaluation aims to find out. 
For example to: 

ascertain results (output, outcome, impact) and assess the effectiveness, 
efficiency, relevance and sustainability of a specific development intervention, 

provide findings, conclusions and recommendations with respect to a specific 
development intervention in order to draw lessons for future design and 
implementation. 

2.3 EVALUATION OBJECT AND SCOPE 

The development intervention being evaluated (the evaluation object) is clearly defined, 
including a description of the intervention logic or theory. The evaluation scope defines 
the time period, funds spent, geographical area, target groups, organisational set-up, 
implementation arrangements, policy and institutional context and other dimensions 
to be covered by the evaluation. Discrepancies between the planned and actual 
implementation of the development intervention are identified. 


2.4 EVALUABILITY 

The feasibility of an evaluation is assessed. Specifically, it should be determined 
whether or not the development intervention is adequately defined and its results 
verifiable, and if evaluation is the best way to answer questions posed by policy 
makers or stakeholders. 

2.5 STAKEHOLDER INVOLVEMENT 

Relevant stakeholders are involved early on in the evaluation process and given the 
opportunity to contribute to evaluation design, including by identifying issues to be 
addressed and evaluation questions to be answered. 

2.6 SYSTEMATIC CONSIDERATION OF JOINT EVALUATION 

To contribute to harmonisation, alignment and an efficient division of labour, donor 
agencies and partner countries systematically consider the option of a joint evaluation, 
conducted collaboratively by more than one agency and/or partner country. 

Joint evaluations address both questions of common interest to all partners and 
specific questions of interest to individual partners. 

2.7 EVALUATION QUESTIONS 

The evaluation objectives are translated into relevant and specific evaluation questions. 
Evaluation questions are decided early on in the process and inform the development 
of the methodology. The evaluation questions also address cross-cutting issues, such 
as gender, environment and human rights. 

2.8 SELECTION AND APPLICATION OF EVALUATION CRITERIA 

The evaluation applies the agreed DAC criteria for evaluating development assistance: 
relevance, efficiency, effectiveness, impact and sustainability. The application of these 
and any additional criteria depends on the evaluation questions and the objectives 
of the evaluation. If a particular criterion is not applied and/or any additional criteria 
added, this is explained in the evaluation report. All criteria applied are defined in 
unambiguous terms. 


2.9 SELECTION OF APPROACH AND METHODOLOGY 

The purpose, scope and evaluation questions determine the most appropriate 
approach and methodology for each evaluation. An inception report can be used to 
inform the selection of an evaluation approach. 

The methodology is developed in line with the evaluation approach chosen. The 
methodology includes specification and justification of the design of the evaluation and 
the techniques for data collection and analysis. The selected methodology answers 
the evaluation questions using credible evidence. A clear distinction is made between 
the different result levels (intervention logic containing an objective-means hierarchy 
stating input, output, outcome, impact). 

Indicators for measuring achievement of the objectives are validated according to 
generally accepted criteria, such as SMART (Specific, Measurable, Attainable, Realistic 
and Timely). Disaggregated data should be presented to clarify any differences between 
sexes and between different groups of poor people, including excluded groups. 

2.10 RESOURCES 

The resources provided for the evaluation are adequate, in terms of funds, staff and 
skills, to ensure that the objectives of the evaluation can be fulfilled effectively. 

2.11 GOVERNANCE AND MANAGEMENT STRUCTURES 

The governance and management structures are designed to fit the evaluation’s 
context, purpose, scope and objectives. 

The evaluation governance structure safeguards credibility, inclusiveness, and 
transparency. Management organises the evaluation process and is responsible for 
day-to-day administration. Depending on the size and complexity of the evaluation, 
these functions may be combined or separate. 

2.12 DOCUMENT DEFINING PURPOSE AND EXPECTATIONS 

The planning and design phase culminates in the drafting of a clear and complete 
written document, usually called “Terms of Reference” (TOR), presenting the purpose, 
scope, and objectives of the evaluation; the methodology to be used; the resources 
and time allocated; reporting requirements; and any other expectations regarding 
the evaluation process and products. The document is agreed to by the evaluation 
manager(s) and those carrying out the evaluation. This document can alternatively be 
called “scope of work” or “evaluation mandate”. 


3.1 EVALUATION TEAM 

A transparent and open procurement procedure is used for selecting the evaluation 
team. 

The members of the evaluation team possess a mix of evaluative skills and thematic 
knowledge. Gender balance is considered and the team includes professionals from 
partner countries or regions concerned. 

3.2 INDEPENDENCE OF EVALUATORS VIS-A-VIS STAKEHOLDERS 

Evaluators are independent from the development intervention, including its policy, 
operations and management functions, as well as intended beneficiaries. Possible 
conflicts of interest are addressed openly and honestly. The evaluation team is able 
to work freely and without interference. It is assured of co-operation and access to all 
relevant information. 

3.3 CONSULTATION AND PROTECTION OF STAKEHOLDERS 

The full range of stakeholders, including both partners and donors, are consulted 
during the evaluation process and given the opportunity to contribute. The criteria for 
identifying and selecting stakeholders are specified. 

The rights and welfare of participants in the evaluation are protected. Anonymity and 
confidentiality of individual informants is protected when requested or as needed. 

3.4 IMPLEMENTATION OF EVALUATION WITHIN ALLOTTED TIME AND BUDGET 

The evaluation is conducted and results are made available to commissioners in a 
timely manner to achieve the objectives of the evaluation. The evaluation is carried out 
efficiently and within budget. Changes in conditions and circumstances are reported 
and un-envisaged changes to timeframe and budget are explained, discussed and 
agreed between the relevant parties. 


3.5 EVALUATION REPORT 

The evaluation report can readily be understood by the intended audience(s) and the 
form of the report is appropriate given the purpose(s) of the evaluation. 

The report covers the following elements and issues: 

3.6 CLARITY AND REPRESENTATIVENESS OF SUMMARY 

A written evaluation report contains an executive summary. The summary provides an 
overview of the report, highlighting the main findings, conclusions, recommendations 
and any overall lessons. 

3.7 CONTEXT OF THE DEVELOPMENT INTERVENTION 

The evaluation report describes the context of the development intervention, 
including: 

policy context, development agency and partner policies, objectives and 
strategies; 

development context, including socio-economic, political and cultural factors; 

institutional context and stakeholder involvement. 

The evaluation identifies and assesses the influence of the context on the performance 
of the development intervention. 

3.8 INTERVENTION LOGIC 

The evaluation report describes and assesses the intervention logic or theory, including 
underlying assumptions and factors affecting the success of the intervention. 




3.9 VALIDITY AND RELIABILITY OF INFORMATION SOURCES 

The evaluation report describes the sources of information used (documents, 
respondents, administrative data, literature, etc.) in sufficient detail so that the 
adequacy of the information can be assessed. The evaluation report explains the 
selection of case studies or any samples. Limitations regarding the representativeness 
of the samples are identified. 

The evaluation cross-validates the information sources and critically assesses the 
validity and reliability of the data. 

Complete lists of interviewees and other information sources consulted are included in 
the report, to the extent that this does not conflict with the privacy and confidentiality 
of participants. 

3.10 EXPLANATION OF THE METHODOLOGY USED 

The evaluation report describes and explains the evaluation methodology and its 
application. In assessing outcomes and impacts, attribution and/or contribution to 
results are explained. The report acknowledges any constraints encountered and how 
these have affected the evaluation, including the independence and impartiality of the 
evaluation. It details the techniques used for data collection and analysis. The choices 
are justified and limitations and shortcomings are explained. 

3.11 CLARITY OF ANALYSIS 

The evaluation report presents findings, conclusions, recommendations and lessons 
separately and with a clear logical distinction between them. 

Findings flow logically from the analysis of the data, showing a clear line of evidence 
to support the conclusions. Conclusions are substantiated by findings and analysis. 
Recommendations and any lessons follow logically from the conclusions. Any 
assumptions underlying the analysis are made explicit. 
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3.12 EVALUATION QUESTIONS ANSWERED 

The evaluation report answers all the questions detailed in the TOR for the evaluation. 
Where this is not possible, explanations are provided. The original questions, as well 
as any revisions to these questions, are documented in the report for readers to be 
able to assess whether the evaluation team has sufficiently addressed the questions, 
including those related to cross-cutting issues, and met the evaluation objectives. 

3.13 ACKNOWLEDGEMENT OF CHANGES AND LIMITATIONS OF THE EVALUATION 

The evaluation report explains any limitations in process, methodology or data, 
and discusses validity and reliability. It indicates any obstruction of a free and open 
evaluation process which may have influenced the findings. Any discrepancies 
between the planned and actual implementation and products of the evaluation are 
explained. 

3.14 ACKNOWLEDGEMENT OF DISAGREEMENTS WITHIN THE EVALUATION TEAM 

Evaluation team members have the opportunity to dissociate themselves from 
particular judgements and recommendations on which they disagree. Any unresolved 
differences of opinion within the team are acknowledged in the report. 

3.15 INCORPORATION OF STAKEHOLDERS’ COMMENTS 

Relevant stakeholders are given the opportunity to comment on the draft report. The 
final evaluation report reflects these comments and acknowledges any substantive 
disagreements. In disputes about facts that can be verified, the evaluators investigate 
and change the draft where necessary. In the case of opinion or interpretation, 
stakeholders’ comments are reproduced verbatim, in an annex or footnote, to the 
extent that this does not conflict with the rights and welfare of participants. 


4.1 TIMELINESS, RELEVANCE AND USE OF THE EVALUATION 

The evaluation is designed, conducted and reported to meet the needs of the intended 
users. Conclusions, recommendations and lessons are clear, relevant, targeted and 
actionable so that the evaluation can be used to achieve its intended learning and 
accountability objectives. The evaluation is delivered in time to ensure optimal use of 
the results. 

Systematic dissemination, storage and management of the evaluation report is ensured 
to provide easy access to all development partners, to reach target audiences, and to 
maximise the learning benefits of the evaluation. 

4.2 SYSTEMATIC RESPONSE TO AND FOLLOW-UP ON RECOMMENDATIONS 

Recommendations are systematically responded to and action taken by the person(s)/ 
body targeted in each recommendation. This includes a formal management response 
and follow-up. All agreed follow-up actions are tracked to ensure accountability for 
their implementation. 

4.3 DISSEMINATION 

The evaluation results are presented in an accessible format and are systematically 
distributed internally and externally for learning and follow-up actions and to ensure 
transparency. In light of lessons emerging from the evaluation, additional interested 
parties in the wider development community are identified and targeted to maximise 
the use of relevant findings. 


OECD DAC Principles for the Evaluation of Development Assistance 
(OECD DAC, 1991) 

OECD DAC Principles for Effective Aid 
(OECD DAC, 1992) 

OECD DAC Glossary of Key Terms in Evaluation and Results Based Management 
(English/ French/ Spanish and other languages, OECD DAC, 2002-2008) 

Evaluation Feedback for Effective Learning and Accountability 
(OECD DAC Network on Development Evaluation, 2001) 

OECD DAC Guidance for Managing Joint Evaluations 
(OECD DAC Network on Development Evaluation, 2006) 

Evaluation Systems and Use, a Working Tool for Peer Reviews and Assessments 
(OECD DAC Network on Development Evaluation, 2006) 
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Quality Standards for Development Evaluation 

The DAC Quality Standards for Development Evaluation provide a guide to good 
practice in development evaluation. They are intended to improve the quality of 
evaluation processes and products and to facilitate collaboration. Built through 
international consensus, the Standards outline the key quality dimensions for 
each phase of a typical evaluation process: defining purpose, planning, designing, 
implementing, reporting, and learning from and using evaluation results. The Standards 
begin with some overall considerations to keep in mind throughout the evaluation 
process. An annex provides references to related OECD DAC development evaluation 
publications. For more information on this publication and the work of the DAC 
Network on Development Evaluation, visit www.oecd.org/dac/evaluationnetwork. 


The full text of this book is available on line via the OECD iLibrary at 
http://dx.doi.org/10. 1 787/19900988 


OECDpublishmg 

www.oecd.org/publishing 


43 2010 06 1 E 



